SWS:lmp 4/12/2004 P0971 EXPRESS MAIL EV324207759US 

HIDING INFORMATION TO REDUCE VISUAL ARTIFACTS 

Related Application Data 
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filed July 2, 2001 (now U.S. Patent No. 6,721,440). The 09/898,901 application is a 
continuation-in-part of U.S. Patent Application No. 09/553,084, filed April 19, 2000 
(now U.S. Patent No. 6,590,996). Each of the above-mentioned patent documents is 
herein incorporated by reference. The present application is also related to U.S. Patent 
Application No. 09/503,881, filed February 14, 2000 (now U.S. Patent No. 6,614,914), 
which is a continuation in part of U.S. Patent Application No. 09/186,962, filed 
November 5, 1998, which is a continuation of application 08/649,419, filed May 16, 
1996, now US Patent 5,862,260. Application No. 08/649,419 is a continuation in part of 
PCT/US96/06618, filed May 7, 1996, U.S. Application No. 08/637,531, filed April 25, 
1996 (now U.S. Patent No. 5,822,436), U.S. Application No. 08/534,005, filed September 
25, 1995 (now U.S. Patent No. 5,832,1 19), and U.S. Application No. 08/436,102, filed 
May 8, 1995 (now U.S. Patent No. 5,748,783). 

Field of the Invention 

[0002] The present invention relates to digital watermarking systems and methods, and 
is particularly illustrated with reference to fragile and low- visibility watermarks. 

Background and Summary of the Invention 
[0003] In color image processing applications, it is useful to understand how humans 
perceive colors. By understanding the human visual system and its sensitivity to certain 
colors, one can more effectively create and manipulate images to create a desired visual 
effect. This assertion is particularly true in image processing applications that 
intentionally alter an image to perform a desired function, like hiding information in an 
image or compressing an image. In digital watermarking, for example, one objective is 
to encode auxiliary information into a signal, such as an image or video sequence, so that 
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the auxiliary information is substantially imperceptible to humans in an output form of 
the signal. 

[0004] Digital watermarking technology, a form of steganography, encompasses a great 
variety of techniques by which plural bits of digital data are hidden in some other object, 
preferably without leaving human-apparent evidence of alteration. 

[0005] Digital watermarking may be used to modify media content to embed a 
machine-readable code into the media content. The media may be modified such that the 
embedded code is imperceptible or nearly imperceptible to the user, yet may be detected 
through an automated detection process. 

[0006] Most commonly, digital watermarking is applied to media signals such as 
images, audio, and video signals. However, it may also be applied to other types of data, 
including documents (e.g., through line, word or character shifting, through texturing, 
graphics, or backgrounds, etc.), software, multi-dimensional graphics models, and surface 
textures of objects. 

[0007] There are many processes by which media can be processed to encode a digital 
watermark. Some techniques employ very subtle printing, e.g., of fine lines or dots, 
which has the effect slightly tinting the media (e.g., a white media can be given a 
lightish-green cast). To the human observer the tinting appears uniform. Computer 
analyses of scan data from the media, however, reveals slight localized changes, 
permitting a multi-bit watermark payload to be discerned. Such printing can be by ink 
jet, dry offset, wet offset, xerography, etc. 

[0008] The encoding of a document can encompass artwork or printing on the 
document, the document's background, a laminate layer applied to the document, surface 
texture, etc. If a photograph or image is present, it too can be encoded. 
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[0009] Printable media - especially for security documents (e.g., banknotes) and 
identity documents (e.g., passports) - is increasingly fashioned from synthetic materials. 
Polymeric films, such as are available from UCB Films, PLC of Belgium, are one 
example. Such films may be clear and require opacification prior to use as substrates for 
security documents. The opacification can be affected by applying plural layers of ink or 
other material, e.g., by gravure or offet printing processes. (Suitable inks are available, 
e.g., from Sicpa Securink Corp. of Springfield, VA.) In addition to obscuring the 
transparency of the film, the inks applied through the printing process form a layer that is 
well suited to fine-line printing by traditional intaglio methods. Such an arrangement is 
more particularly detailed in laid-open PCT publication W098/33758. 

[0010] Digital watermarking systems typically have two primary components: an 
embedding component that embeds the watermark in the media content, and a reading 
component that detects and reads the embedded watermark. The embedding component 
embeds a watermark pattern by altering data samples of the media content. The reading 
component analyzes content to detect whether a watermark pattern is present. In 
applications where the watermark encodes information, the reading component extracts 
this information from the detected watermark. Previously mentioned U.S. Patent 
Application No. 09/503,881, filed February 14, 2000, discloses various encoding and 
decoding techniques. United States Patent Nos. 5,862,260 and 6,122,403 disclose still 
others. Of course, artisans know many other watermarking techniques that may be 
suitably interchanged with the present invention. 

[0011] One form of digital watermarks is a so-called "fragile" watermark. A fragile 
watermark is designed to be lost, or to degrade predictably, when the data set into which 
it is embedded is processed in some manner, such as signal processing, scanning/printing, 
etc. A watermark may be made fragile in numerous ways. One form of fragility relies on 
low watermark amplitude. That is, the strength of the watermark is only marginally 
above the minimum needed for detection. If any significant fraction of the signal is lost, 
as typically occurs in photocopying operations, the watermark becomes unreadable. 
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Another form of fragility relies on the watermark's frequency spectrum. High 
frequencies are typically attenuated in the various sampling operations associated with 
digital scanning and printing. Even a high amplitude watermark signal can be 
significantly impaired, and rendered unreadable, by such photocopying operations. 
(Fragile watermark technology and various applications of such are even further 
disclosed, e.g., in assignee's U.S. Patent Application Nos. 09/234,780, 09/433,104, 
09/498,223, 60/198,138, 09/562,516, 09/567,405, 09/625,577, 09/645,779, and 
60/232,163.). 

[0012] The present invention discloses a new fragile watermarking technique that is 
particularly well suited for color imaging applications. A watermark signal in one color 
plane (or channel) is applied to be out of phase with corresponding watermark signals in 
other color planes (or channels). An effect of the inventive out-of-phase watermarking 
technique is to greatly reduce watermark visibility by canceling perceived luminance 
change in local areas throughout the image. The disclosed watermark is also fragile, 
since signal-processing operations that combine the out-of-phase color channel with the 
other channels cancels the watermark signal. 

[0013] The foregoing and other features and advantages of the present invention will be 
more readily apparent from the following detailed description, which proceeds with 
reference to the accompanying drawings. 

Brief Description of the Drawings 

[0014] Fig. 1 is a diagram of a color space depicting how to scale a color vector to 
black to effect a change in luminance. 

[0015] Fig. 2 is a diagram of a color space depicting how to scale a color vector to 
white to effect a change in luminance. 
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[0016] Fig. 3a is a diagram illustrating color data for a mid-gray image patch. 

[0017] Fig. 3b illustrates the color data of Fig. 3a, embedded with a digital watermark 
signal. 

[0018] Fig. 4 is a flow diagram illustrating an embedding process. 

[0019] Figs. 5 and 6 are diagrams regarding obtaining luminance values. 

[0020] Fig. 7 is a flow diagram illustrating a detection method. 

[0021] Fig. 8 is a flow diagram illustrating a detection method of data in a RGB color 
space. 

[0022] Fig. 9 illustrates approximating a color saturation value in an RGB color space. 

[0023] Fig. 10 is a flow diagram illustrating a luminance collection method according 
to the present invention. 

[0024] Figs. 1 1 and 12 are diagrams illustrating relative threshold values for the method 
shown in Fig. 8. 

[0025] Fig. 13 is a diagram illustrating RGB and CMY common color gamuts, and out- 
of-gamut colors. 

Detailed Description 

Introduction 

[0026] A watermark can be viewed as an information signal that is embedded in a host 
signal, such as an image, audio, video or some other media content. Watermarking 
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systems typically include the following components: 1) an embedder that inserts a 
watermark signal in the host signal to form a combined signal; 2) a detector that 
determines the presence and orientation of a watermark in a potentially corrupted version 
of the combined signal; and 3) a reader that extracts a watermark message from the 
combined signal. In some implementations, the detector and reader are combined. 

[0027] To encode a message, the watermark encoder analyzes and selectively adjusts 
the host signal to give it attributes that correspond to a desired message symbol or 
symbols to be encoded. There are many signal attributes that may encode a message 
symbol, such as a positive or negative polarity of signal samples or a set of samples, a 
given parity (odd or even), a given difference value or polarity of the difference between 
signal samples (e.g., a difference between selected spatial intensity values or transform 
coefficients), a given distance value between watermarks, a given phase or phase offset 
between different watermark components, a modulation of the phase of the host signal, a 
modulation of frequency coefficients of the host signal, a given frequency pattern, a 
given quantizer (e.g., in Quantization Index Modulation), etc. 

[0028] The structure and complexity of a watermark signal can vary significantly, 
depending on the application. For example, the watermark may be comprised of one or 
more signal components, each defined in the same or different domains. Each 
component may perform one or more functions. Two primary functions include acting as 
an identifier to facilitate detection and acting as an information carrier to convey a 
message. In addition, components may be located in different spatial or temporal 
portions of the host signal, and may carry the same or different messages. 

[0029] The host signal can vary as well. The host is typically some form of multi- 
dimensional media signal, such as an image, audio sequence or video sequence. In the 
digital domain, each of these media types is represented as a multi-dimensional array of 
discrete samples. For example, a color image has spatial dimensions (e.g., its horizontal 
and vertical components), and color space dimensions (e.g., CMYK, YUV or RGB). 
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Some signals, like video, have spatial and temporal dimensions. Depending on the needs 
of a particular application, the embedder may insert a watermark signal that exists in one 
or more of these dimensions. 

[0030] In the design of the watermark and its components, developers are faced with 
several design issues such as: the extent to which the mark is impervious to jamming and 
manipulation (either intentional or unintentional); the extent of imperceptibility; the 
quantity of information content; the extent to which the mark facilitates detection and 
recovery, and the extent to which the information content can be recovered accurately. 

[0031] For certain applications, such as copy protection or authentication, the 
watermark should be difficult to tamper with or remove by those seeking to circumvent 
it. To be robust, the watermark should withstand routine manipulation, such as data 
compression, copying, linear transformation, flipping, inversion, etc., and intentional 
manipulation intended to remove the mark or make it undetectable. Some applications 
require the watermark signal to remain robust through digital to analog conversion (e.g., 
printing an image or playing music), and analog to digital conversion (e.g., scanning the 
image or digitally sampling the music). In some cases, it is beneficial for the 
watermarking technique to withstand repeated watermarking. 

[0032] For other applications, such as forensic tracking, counterfeit detection, etc., the 
watermark should degrade predictably under routine manipulation. Such watermarks are 
refereed to generally as "fragile" watermarks, as discussed above. 

[0033] A variety of signal processing techniques may be applied to address some or all 
of these design considerations. One such technique is referred to as spreading. 
Sometimes categorized as a spread spectrum technique, spreading is a way to distribute a 
message into a number of components (chips), which together make up the entire 
message. Spreading makes the mark more impervious to jamming and manipulation, and 
makes it less perceptible. 
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[0034] Another category of signal processing technique is error correction and 
detection coding. Error correction coding is useful to reconstruct the message accurately 
from the watermark signal. Error detection coding enables the decoder to determine 
when the extracted message has an error. 

[0035] Another signal processing technique that is useful in watermark coding is called 
scattering. Scattering is a method of distributing the message or its components among 
an array of locations in a particular transform domain, such as a spatial domain or a 
spatial frequency domain. Like spreading, scattering makes the watermark less 
perceptible and more impervious to manipulation. 

[0036] Yet another signal processing technique is gain control. Gain control is used to 
adjust the intensity of the watermark signal. The intensity of the signal impacts a number 
of aspects of watermark coding, including its perceptibility to the ordinary observer, and 
the ability to detect the mark and accurately recover the message from it. 

[0037] Gain control can impact the various functions and components of the watermark 
differently. Thus, in some cases, it is useful to control the gain while taking into account 
its impact on the message and orientation functions of the watermark or its components. 
For example, in a watermark system described below, the embedder calculates a different 
gain for orientation and message components of an image watermark. 

[0038] Another useful tool in watermark embedding and reading is perceptual analysis. 
Perceptual analysis refers generally to techniques for evaluating signal properties based 
on the extent to which those properties are (or are likely to be) perceptible to humans 
(e.g., listeners or viewers of the media content). A watermark embedder can take 
advantage of a Human Visual System (HVS) model to determine where to place an image 
watermark and how to control the intensity of the watermark so that chances of 
accurately recovering the watermark are enhanced, resistance to tampering is increased, 
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and perceptibility of the watermark is reduced. Similarly, audio watermark embedder can 
take advantage of a Human Auditory System model to determine how to encode an audio 
watermark in an audio signal to reduce audibility. Such perceptual analysis can play an 
integral role in gain control because it helps indicate how the gain can be adjusted 
relative to the impact on the perceptibility of the mark. Perceptual analysis can also play 
an integral role in locating the watermark in a host signal. For example, one might design 
the embedder to hide a watermark in portions of a host signal that are more likely to mask 
the mark from human perception. 

[0039] Various forms of statistical analyses may be performed on a signal to identify 
places to locate the watermark, and to identify places where to extract the watermark. 
For example, a statistical analysis can identify portions of a host image that have noise- 
like properties that are likely to make recovery of the watermark signal difficult. 
Similarly, statistical analyses may be used to characterize the host signal to determine 
where to locate the watermark. 

[0040] Each of the techniques may be used alone, in various combinations, and in 
combination with other signal processing techniques. 

[0041] In addition to selecting the appropriate signal processing techniques, the 
developer is faced with other design considerations. One consideration is the nature and 
format of the media content. In the case of digital images, for example, the image data is 
typically represented as an array of image samples. Color images are represented as an 
array of color vectors in a color space, such as RGB or YUV. The watermark may be 
embedded in one or more of the color components of an image. In some 
implementations, the embedder may transform the input image into a target color space, 
and then proceed with the embedding process in that color space. 
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Color Image Processing 

[0042] In image processing applications, it is sometimes useful to be able to change the 
colors of an image while reducing the visibility of these changes. Image watermarking is 
one application where it is desirable to alter image samples to encode information in a 
manner that is readily recoverable by an automated process, yet substantially 
imperceptible to human visual perception. Often, the aim of watermark encoding is to 
maximize a watermark signal without significantly affecting image quality. Since the eye 
is more sensitive to changes in memory colors such as flesh tones or blue sky, it is 
beneficial to have a method of selectively controlling strength of a watermark in certain 
color regions. Previously mentioned U.S. Patent Application No. 09/553,084 discloses 
various methods for such. 

[0043] In one embodiment, a watermark encodes auxiliary information in an image by 
making changes to image samples. A color-masking framework maps a change in an 
image sample attribute to an equivalent yet less perceptible change in the color values of 
that image sample. This mapping can be used to obtain equal perceptual watermark 
changes to image samples in other areas of color space and to apply the change in the 
least visible color channels. 

[0044] While the implementation details of watermark encoding schemes vary 
significantly, a class of watermarking schemes can be modeled as an array of changes to 
luminance values of a host image. The host image comprises an array of color vectors 
(e.g., an array of color such as RGB, CMY, CMYK, etc). The image sample may be 
represented as a vector between black and the pixel color value. To encode a watermark, 
the luminance of the image sample may be increased or decreased as shown in Fig. 1. 
Fig. 1 shows a 3-dimensional color space with Cyan (C), Magenta (M) and Yellow (Y) 
axes. The bold axis between black and white represents luminance. To make an 
equivalent luminance change in an image sample of a given color vector (CI, Ml, Yl), 
one may make a corresponding scale to black as shown. 
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[0045] An alternative method of obtaining the same luminance change is to scale the 
image sample like a vector between white and the sample's color value as shown in Fig. 
2. To make an equivalent luminance change, one may make a corresponding scale to 
white as shown. 

[0046] By using the scale to white method for colors with high yellow content such as 
yellow, red and green, and scale to black for blue, cyan and magenta a lower visibility 
watermark can be encoded with the same detectability. 

[0047] Once the color vector entries are established, each of the entries is associated 
with a set of scale factors. The set includes a scale factor for each color component. The 
specific color components in the implementation depend on the color format of the 
image. For example, images in an RGB format have scale factors for each of the R, G 
and B color components. Similarly, images in a CMY format have scale factors for each 
of the C, M and Y components of each table entry. The scale factors for each entry are 
derived by rewriting the above mathematical expression and solving for each color's 
scale factor as a function of the known color component values. 

Low Visibility Watermarks Using an Out-of-Phase Color 

[0048] Three-color components, such as Red, Green, Blue (RGB) or Luminance, color 
component "a" and color component "b" (Lab), can be combined to uniquely identify a 
particular color. In many cases, more than three-color components can be combined to 
specify (or approximate) the same particular color. Typically, four (4) color components, 
e.g., cyan, magenta, yellow and black (CMYK) are used in printing processes. Extra 
colors, sometimes referred to as "spot colors," can be added for more accurate color 
reproduction. A company logo, might include a particular shade of green, which is 
printed with a corresponding green ink (e.g., a green spot color). High fidelity color 
printing often uses more than four (4) color components. These additional color 
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components expand the gamut of printing colors for very high quality printing, such as 
fine art reproduction. Security printing (e.g., bank notes, financial documents, 
certificates, etc.) also uses a plurality of inks, with fine lines, such as intaglio. 

[0049] Printing processes with more than three (3) inks (or color component 
dimensions) have a built in redundancy, since more than three (3) color components (e.g., 
CMYK) are used to specify a color at a particular point in an image. This implies that 
many different combinations of four (4) or more printing inks (or components) can be 
used to represent a three (3)-component color. By way of example only, a point in the 
CMY color space (e.g., 61% cyan, 50% magenta, and 48% yellow) can be represented (or 
approximated) by a corresponding point in the CMYK color space (e.g., 51% cyan, 40% 
magenta, 38% yellow, and 22% black). This same 3-channel color point can also be 
represented in the CMYK color space as 32% cyan, 23% magenta, 22% yellow, and 43% 
black. Of course, this is but one of many possible color combinations. One aspect of the 
present invention utilizes this color redundancy, to reduce the human visibility of a digital 
watermark. For example, a watermark signal in one color channel, can be effectively 
counteracted (e.g., cancelled) in the other color (or black) channels, while still obtaining 
the desired color. 

[0050] Another aspect of the present invention provides a fragile watermark. The 
watermark's fragility is due, at least in part, to its small-scale spatial variation within a 
media signal, making the watermark ideal to combat typical counterfeiting operations 
such as scanning/printing, digital manipulation and photocopying. Of course, the process 
described for "CMYK" media below, could also be applied in a similar manner to other 
multi-color plane printing processes. 

Watermark Embedding 

[0051] Media is embedded with a watermark signal. Of course, the media may 
correspond to an image, digital image, photograph, video frame, graphic, etc., and in 
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some cases, may even include a physical object such as a document, banknote, postage 
stamp, etc. Typically, a watermark signal has at least one component, which when 
embedded in the media corresponds to (or affects) various areas (or pixels) in the media. 
In the case of an area (or pixel) represented in a color space, the watermark signal 
component can be added to (or subtracted from) the point in all or some of the color 
dimensions. 

[0052] Consider Fig. 3a, where the dash/dot C,M, Y and K lines represent respective 
cyan, magenta, yellow and black color dimensions for a line in a mid-gray patch of a 
media signal (e.g., a line in a picture, image, document, etc.). Fig. 3b illustrates the 
media of Fig. 3a, which has been embedded with a watermark signal. The watermark 
signal is preferably applied to each of the color component dimension (C, M, and Y). In 
Figs. 3a and 3b, the M and Y channels are represented by one signal, since these color 
components can be approximately equal, but separate signals for gray. Of course, it is 
not necessary for these components to be equal, and in many cases the yellow and 
magenta components are not equal. The illustrated embedded "bumps" in Fig. 3b 
represent the watermark signal, e.g., upward or downward signal adjustments in relation 
to the respective color channel at given points over the media line. For the K dimension 
(or channel), the watermark signal is preferably embedded to be out-of-phase with the 
CMY channels. Most preferably, the K channel is approximately 180 degrees out-of- 
phase (e.g., inverted) with the watermark signals in the CMY color channels, as shown in 
Fig. 3b. 

[0053] With reference to Fig. 4, one out-of-phase embedding method generates (or 
determines) a watermark signal as shown in step SI. The watermark signal is embedded 
in the CMY channels in step S2. The inverse of the watermark signal is calculated in step 
S3, and such inverse signal is embedded in the K channel in step S4. Of course, the order 
of such steps is not critical. For example, a watermark signal can be determined, and an 
inverse calculated. The various color component dimensions can then be embedded. In 
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another embodiment, the K channel is embedded with a watermark signal. An inverse 
signal is calculated, and the CMY channels are embedded with the inverse signal. 

[0054] Such an inventive watermarking scheme greatly reduces watermark visibility. 
Since the watermark signal for the K channel is applied approximately 180 degrees out of 
phase, when compared to the respective changes applied to the CMY channels, the 
watermark visibility is greatly reduced. The visibility reduction is produced by the 
effective cancellation of perceived luminance changes, when the CMYK image is viewed 
or printed. Indeed, combining an inverted watermark signal "tweak" or "bump" in a K 
channel, with a corresponding non-inverted watermark signal tweak in the CMY channels 
effectively cancels an overall perceived luminance change - effectively reducing 
visibility of the digital watermark. 

Watermark Detection 

[0055] Another aspect of the present invention is a detection method and system for 
detecting an out-of-phase, color component-based digital watermark. Consider a first 
embodiment as depicted in Figs. 5 and 6. A media signal is analyzed as follows. 
Initially, luminance values are determined for CMY and K color planes. The CMY 
luminance can be computed as a properly weighted sum of the cyan, magenta and yellow 
primary components. For example: Lum = .3C + .6M +.1 Y. (Of course, as will be 
appreciated by those skilled in the art, other weighting coefficients can be used to 
determine a CMY luminance.). So for a given point (or pixel), a luminance value is 
determined for the CMY color dimensions. Similarly, as shown in Fig. 6, a luminance 
value for K is determined. In one embodiment, the K luminance equals the value of the 
K component. 



-14- 



SWS:lmp 4/12/2004 P0971 



EXPRESS MAIL EV324207759US 



[0056] The detection process is further described with respect to Fig. 7. In step S10, 
CMY and K are converted to approximate luminance values per pixel (or area): 

CMY => lunicmy 
K => lunik , 

where lumc my is the luminance of CMY and lum k is the luminance of K. In step SI 1, 
lunik is subtracted from lumc my : 

Luminal = lum cmy - lumk. 

[0057] The step SI 1 subtraction operates to help reduce image content, and to reinforce 
the watermark signal by effectively adding the K watermark signal value to the CMY 
watermark signal, since the K watermark signal is the inverse of the CMY channel 
signals. 

[0058] As shown in step SI 2, Lumfi na i can be analyzed for watermark detection. 
Fragile Watermark 

[0059] An out-of-phase watermark is fragile since a signal processing operation that 
combines the K channel with the CMY channel effectively cancels the watermark signal. 
Conversion to other color spaces similarly degrades the watermark signal. Take a typical 
scan/print process for example. Digital scanners typically convert scanned images into a 
RGB color scheme. Scanning an out-of-phase embedded CMYK image degrades the 
embedded watermark due to the combination of K with CMY in a local area. When the 
RGB image representation is printed, the watermark signal is difficult to detect, 
particularly with a low resolution RGB scan. Similarly, other conversions, such as to a 
Lab color space, degrade the out-of-phase watermark due to the combination of K with 
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CMY throughout local areas. Nevertheless, the watermark signal is detectable with 
CMYK data as described above with respect to Fig. 7. 

[0060] A fragile watermark has utility in many applications. Take counterfeiting, for 
example. The inventive fragile watermark is embedded in original CMYK media. If the 
media is copied, the embedded fragile watermark is either lost or degrades predictable. 
The copy is recognized as a copy (or counterfeit) by the absence or degradation of the 
fragile watermark. Fragile watermarks can also be used in conjunction with other 
watermarks, such as robust watermarks. The fragile watermark announces a copy or 
counterfeit by its absence or degradation, while the other robust watermark identifies 
author, source, links and/or conveys metadata or other information, etc. In other 
embodiments, a fragile watermark is an enabler. For example, some fragile watermark 
may include plural-bit data that is used to enable a machine, allow access to a secure 
computer area, verify authenticity, and/or link to information. This plural-bit data is lost 
or sufficiently degrades in a copy, preventing the enabling functions. Other fragile 
watermark applications are discussed in the U.S. patent applications incorporated above. 

High Resolution Scan of Watermarked Image 

[0061] Oftentimes, CMYK data may not be available for a detection process. For 
example, a watermarked CMYK image may be optically scanned with a scanner that 
converts the CMYK data into a different color space, such as to RGB. A high resolution 
RGB scan may nevertheless be used to recover an estimation of the watermark signal, 
which would be otherwise undetectable with a low RGB resolution scan. In this case, 
pixels can be assigned color or K values to generate respective color and K planes. A 
final luminance value can be determined from these planes. Consider the following 
method, as shown in Fig. 8, for a high resolution RGB scan (e.g., about 8 times the screen 
ruling or more) of a CMYK image. 
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[0062] In step S20, a color saturation (ColSat) value is determined for each RGB pixel. 
ColSat can be calculated by dropping a perpendicular line from a measured RGB value 
(e.g., Ro, G 0 , B 0 , as shown in Fig. 9) to the RGB luminance axis. This color saturation 
calculation alternatively can be approximated by: 

ColSat = max(RGB) - min(RGB), 

where max(RGB) is determined by taking the maximum of the red, green and blue values 
Ro, G 0 , B 0 , and min(RGB) is determined by taking the minimum of the red, green and 
blue values Ro, G 0 , B 0 . Of course, other known methods for determining a color 
saturation value are suitably interchangeable with this step. 

[0063] In step S22, preferably for each RGB pixel (or pixel area), it is determined 
whether ColSat < T co i, where T co i is a predetermined threshold color saturation, e.g., 
based on a scanner calibration, detection sensitivity and/or other threshold number. For 
example, a pure cyan pixel will typically correspond to a (0, 100, 100) RGB value. 
However, a scanner may read this pixel as a (20, 80, 80) RGB value. The threshold value 
T C oi can be set to allow for such scanner sensitivity. T co i can also be adjusted to reflect 
acceptable tolerances. If ColSat < T co i, flow continues to step S24, where it is determined 
whether the maximum measured (e.g., determined, scanned, calculated, etc.) pixel value 
is < Ti . Here, Ti is a predetermined pixel value, e.g., based on scanner calibration, 
detection sensitivity and/or other threshold numbers. Fig. 1 1 illustrates a graphical 
relationship that shows digital threshold and scanner pixel values on a scale between 
Black (0) and white (255). Value Sb is a scanner measured (or designated) black, which 
due to scanner sensitivity and/or characteristics is generally offset from black (0). 
Similarly, SCMY is a scanner measured CMY overprint (e.g., a 100% overprint). If the 
maximum pixel value is < Ti the pixel is designated as black (K) in step S26. A 
corresponding pixel range is shown graphically in Fig. 1 1, as the values less than Ti. As 
discussed, Ti can be selected based on scanner characteristics. In one embodiment, Tj is 
selected to have a value midway between scanner-measured black (Sb), and scanner 
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measured CMY overprint (Scmy). Of course, Ti can be adjusted from this midpoint value 
to accommodate sensitivity requirements and/or scanner characteristics. (A maximum 
pixel value can be chosen in a number of known techniques, such as selecting the color 
component of a pixel, e.g., if a measured pixel corresponds to (20, 70, 80), the color 
component 80 comprises the maximum pixel value.). 

[0064] If the max pixel value is not < Ti, the pixel value is compared against another 
threshold value. In step S28, it is determined whether the max pixel value is < T 2 . If so, 
the pixel is designated as color in step S30. Returning again to Fig. 1 1, the corresponding 
range of pixel values falls between Ti and T 2 . The threshold value T 2 can be selected 
based on scanner characteristics, or based on sensitivity requirements. In one 
embodiment, T 2 is selected to have a value midway between Scmy and white (255). Of 
course, this value can be adjusted based on sensitivity need and/or scanner characteristics. 

[0065] If the max pixel value is not < T 2 , the pixel value is designated as white in step 
S32. The corresponding pixel range lies between T 2 and white (255). 

[0066] If in step S22, it is determined that ColSat is not < T co i, flow continues to step 
S34, where it is determined whether the max pixel value is < T 3 . If so, the pixel is 
designated as a color pixel in step S36. Here, T 3 is a predetermined pixel value. In one 
embodiment, T3 is selected to have a value midway between a scanner measured (or 
determined) yellow value and white (255). Of course, this value can be adjusted based 
on sensitivity requirements and/or scanner characteristics. Otherwise, the pixel value is 
designated white in step S3 8. This relationship is shown in Fig. 12, where Sc, Sm and Sy 
corresponding with scanner measured (or determined) cyan, magenta and yellow values. 

[0067] Color and K planes can be constructed once each RGB color pixel is designated 
as a color, white or K pixel. 
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[0068] To create the K plane, pixels designated as black are turned "on/' while the 
pixels that are designated as white or color are turned "off." In one embodiment, the 
respective "off pixels are masked. In another embodiment, the off pixels (or 
alternatively the on pixels) are flagged or otherwise marked to indicate their designation 
and/or inclusion/non-inclusion. 

[0069] Similarly, to create the color plane, pixels designated as "white" or "black" are 
turned off, while the rest of the pixels (e.g., the color pixels) are turned on. 

[0070] The pixel (or area) values are summed for the color plane to obtain a low 
resolution (LR) color luminance (lunicoior) per pixel (or per area). Similarly, the pixel 
values are summed for the black plane to obtain a LR black luminance (lum K ). A final 
luminance value (lumf, na i) for each pixel (or area) can be determined from: 

lumfmai = lunicoior - lum k . 

The lumbal value can be passed into a watermark detection process. 

[0071] Fig. 10 illustrates one possible summing method for each color (color and K) 
plane according to a preferred embodiment to achieve the above color and black 
luminance values. 

[0072] In step S40, each of the luminance values within a predetermined window (or 
pixel area) are summed. The window may be an n x n window, or and n x m window, 
where n and m are integers. In a preferred embodiment, pixels values with an 8 X 8 pixel 
window are summed. The resulting summation value is preferably saved for comparison 
against the other plane (color or K), to determine lumfj na i as discussed above. 

[0073] In step S42, the window location with respect to the color (or K) plane is 
repositioned (e.g., the window is a sliding window). To illustrate, if a first window 
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frames the first n x n pixels in a color (or K) plane, the second window is adjusted to 
cover a new area, or an overlapping area. In the preferred embodiment, the window 
slides right (or horizontally) by four (4) pixels (e.g., on a first slide, the window now 
covers the 5 th - 12 th pixels X 8). The luminance values within this second window are 
added in step S44. This value is preferably saved for comparison against the other color 
plane (or K plane), to determine lumfj na i as discussed above. 

[0074] The method determines if the window is at the end of a line (e.g., at the end of a 
plane edge) in step S46. If not, flow continues to step S42, where the window location is 
again repositioned. Otherwise, it is determined whether the entire plane has been 
analyzed in step S48. If so, the method ends. Otherwise, the window location is 
repositioned in step S50. The step S50 location adjustment preferably moves the window 
location down (or vertically) with respect to the plane to cover a new area or an 
overlapping area. In the preferred embodiment, the window is shifted down by 4 pixels. 
At this step, it is important to note that the new window location need not be realigned at 
the staring point (e.g., the top-left corner of the plane), but may be shifted down at the 
right plane edge. The window may then slide right to left. Of course, the step S50 
alignment may locate the window below the original starting point (e.g., the left plane 
edge), with the window sliding left to right. Flow continues from step S50 to S40. 

[0075] This process is preferably carried out for each of the color and K planes. The 
resulting area black luminance values are subtracted from the corresponding resulting 
color luminance values to achieve the final luminance value. This final luminance value 
(for each location area) can be analyzed to detect the watermark signal. 

Embedding in Out-of-Range Colors 

[0076] Another inventive fragile watermarking technique embeds watermark data in 
out-of-range colors. A color gamut defines a range of colors. Different color schemes 
(e.g., RGB and CMY) generally include a unique color gamut. Such color schemes will 
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most certainly have overlapping color gamuts (or ranges), and unique (or out of gamut) 
color ranges, as shown in Fig. 13. 

[0077] Differences in gamut between color models can be used to indicate that a 
transformation (or copy) has occurred. Consider security printers, which often select 
inks that lie outside the common color gamuts of capture devices (RGB) when printing 
documents. Printing with such out-of-range (or out of gamut) colors makes 
counterfeiting even more difficult. Consider a document that is printed with some dark 
blues & violets in the CMYK space, which are out of gamut for the RGB space. When a 
scanner scans the CMYK document, it typically converts the scanned image into the 
RGB space. Such processing looses the dark blues and violets in the conversion. An 
educated inspector can identify a counterfeit document by looking for the presence (or 
absence) of certain colors. 

[0078] The inventive fragile watermark utilizes out-of-range color gamuts. Take the 
example given above. Dark blues & violets in the CMY (or CMYK) space are out of 
gamut with respect to the RGB space. Accordingly, a mask (or color spectral analysis) is 
used to identify dark blues and violets in a media signal. These areas are used (e.g., 
masked) as the areas for embedding watermark signals. The watermark is detectable with 
a CMY detection process. However, if the document is scanned with a RGB element 
scanner, the watermark is generally lost. As discussed above, conversion from CMYK to 
RGB fails to accurately convert out-of-phase colors. In this case, the dark blues and 
violets are out-of-gamut. Accordingly, since the watermark signal is embedded in the 
out-of-gamut colors, it is lost (or predictably degraded) during the conversion from CMY 
to RGB. 

[0079] Consider the possibilities of such a fragile watermark. One can track, trace, 
detect counterfeits, control enabling functions, and many, many more application, like 
those discussed above. 
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Concluding Remarks 

[0080] The foregoing are just exemplary implementations of the present invention. It 
will be recognized that there are a great number of variations on these basic themes. The 
foregoing illustrates but a few applications of the detailed technology. There are many 
others. 

[0081] It should be appreciated that the order of steps in the Figure 4, 8, and 10 flow 
charts can be reordered without deviating from the scope of the present invention. For 
example, in Fig. 4, the K channel could be embedded first, and then the CMY channels. 
Or the K channel could he embedded concurrently with or in between the color channels. 
In Fig. 8, instead of basing the step S22 decision on color saturation, the step S22 
decision could be based on whether the pixel value is above or below one or all of the 
predetermined thresholds. The color saturation could be analyzed in subsequent steps. 
Also, the signs of the decisions can be reversed, which will respectively reverse the 
decision tree branches, and color ranges. Also, decisions could be based on whether a 
pixel value is less than, or equal to a threshold value. Moreover, instead of a maximum 
pixel value, an average pixel value, or lower pixel value could be used. In Fig. 10, the 
window could alternatively be first vertically repositioned, and then horizontally. The 
window can also be repositioned on a random, or pseudo-random basis. The window size 
may also be varied. Also whereas the Figs. 3a and 3b illustrate a mid-gray patch, the 
present invention is not so limited. Indeed the scope of the present invention covers any 
set of 2 or more primary colors. 

[0082] Preferably, an out-of phase watermark signal is embedded 180 degrees out of 
phase with corresponding channels. However, some cancellation will still be achieved if 
the signal is approximately 180 degrees, for example, in a range of ± 0-20 degrees off of 
the 180-degree mark. 
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[0083] The section headings in this application are provided merely for the reader's 
convenience, and provide no substantive limitations. Of course, the disclosure under one 
section heading may be readily combined with the disclosure under another section 
heading. 

[0084] To provide a comprehensive disclosure without unduly lengthening this 
specification, the above-mentioned patents and patent applications are hereby 
incorporated by reference. The particular combinations of elements and features in the 
above-detailed embodiments are exemplary only; the interchanging and substitution of 
these teachings with other teachings in this application and the incorporated-by-reference 
patents/applications are also contemplated. 

[0085] The above-described methods and functionality can be facilitated with computer 
executable software stored on computer readable media, such as electronic memory 
circuits, RAM, ROM, magnetic media, optical media, memory sticks, hard disks, 
removable media, etc., etc. Such software may be stored and executed on a general 
purpose computer, or on a server for distributed use. Data structures representing the 
various luminance values, summations, out-of-phase embedded signals, embedded color 
planes, color signals, data signals, luminance signals, etc., may also be stored on such 
computer readable media. Also, instead of software, a hardware implementation, or a 
software-hardware implementation can be used. 

[0086] In view of the wide variety of embodiments to which the principles and features 
discussed above can be applied, it should be apparent that the detailed embodiments are 
illustrative only and should not be taken as limiting the scope of the invention. Rather, 
we claim as our invention all such modifications as may come within the scope and spirit 
of the following claims and equivalents thereof. 
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